{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cHgpJN1uzD8B"
      },
      "source": [
        "# 训练教程\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2meQHBlHxcsi"
      },
      "source": [
        "## Install Dependencies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "quRXOPaZwmwz"
      },
      "outputs": [],
      "source": [
        "!pip install transformers==4.28.1\n",
        "!pip install git+https://github.com/huggingface/peft.git@13e53fc\n",
        "!pip install datasets\n",
        "!pip install sentencepiece\n",
        "!pip install deepspeed"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ji21WFqexASI"
      },
      "source": [
        "## 克隆代码\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dulrlPMexFNN"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/IMOSR/Media-LLaMA.git"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pqH_h_ZAz4_e"
      },
      "source": [
        "## 指令微调 Alpaca-7B\n",
        "\n",
        "参考chinese llama的博客 https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/SFT-Script 设置对应的参数"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IhrAVNUKSw9_"
      },
      "outputs": [],
      "source": [
        "# 对应目录记得改成自己的\n",
        "\n",
        "!cd [脚本地址]/train && torchrun --nnodes 1 --nproc_per_node 1 run_clm_sft_with_peft.py \\\n",
        "    --deepspeed ds_zero2_no_offload.json \\\n",
        "    --model_name_or_path decapoda-research/llama-7b-hf \\\n",
        "    --tokenizer_name_or_path ziqingyang/chinese-alpaca-lora-7b \\\n",
        "    --dataset_dir [数据地址]/tiktok_data \\\n",
        "    --validation_split_percentage 0.001 \\\n",
        "    --per_device_train_batch_size 1 \\\n",
        "    --do_train \\\n",
        "    --fp16 \\\n",
        "    --seed $RANDOM \\\n",
        "    --max_steps 10000 \\\n",
        "    --lr_scheduler_type cosine \\\n",
        "    --learning_rate 1e-4 \\\n",
        "    --warmup_ratio 0.03 \\\n",
        "    --weight_decay 0 \\\n",
        "    --logging_strategy steps \\\n",
        "    --logging_steps 10000 \\\n",
        "    --save_strategy steps \\\n",
        "    --save_total_limit 2 \\\n",
        "    --save_steps 10000 \\\n",
        "    --gradient_accumulation_steps 1 \\\n",
        "    --preprocessing_num_workers 8 \\\n",
        "    --max_seq_length 512 \\\n",
        "    --output_dir [输出地址] \\\n",
        "    --overwrite_output_dir \\\n",
        "    --ddp_timeout 30000 \\\n",
        "    --logging_first_step True \\\n",
        "    --torch_dtype float16 \\\n",
        "    --peft_path [peft 模型的地址] \\\n",
        "    --gradient_checkpointing \\\n",
        "    --ddp_find_unused_parameters False"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 合并模型\n",
        "参考Chinese llama的文档：https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E5%9C%A8%E7%BA%BF%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2"
      ],
      "metadata": {
        "id": "FapMcYEC-fIh"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VnLFI6p60K_5"
      },
      "outputs": [],
      "source": [
        "!python merge/merge_llama_with_chinese_lora.py \\\n",
        "    --base_model  decapoda-research/llama-7b-hf\\\n",
        "    --lora_model ziqingyang/chinese-llama-plus-lora-7b,[你训练完的模型的地址]/sft_lora_model \\\n",
        "    --output_type pth \\\n",
        "    --output_dir /content/drive/MyDrive/gitcode/sample_data/llama-combined-ch-7"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# llama.cpp量化部署\n"
      ],
      "metadata": {
        "id": "qdSoX7EQ_TFa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 克隆代码和编译"
      ],
      "metadata": {
        "id": "MGa2cu_O_Z8A"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/ggerganov/llama.cpp"
      ],
      "metadata": {
        "id": "ZrUrpyJe_fPJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eeYQtt8dZRI7"
      },
      "outputs": [],
      "source": [
        "\n",
        "!cd /content/drive/MyDrive/gitcode/llama.cpp && make"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 模型转换为ggml格式（FP16）"
      ],
      "metadata": {
        "id": "PcnyvV8fAZyg"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Imj6Q3Pda_0o"
      },
      "outputs": [],
      "source": [
        "!cd /content/drive/MyDrive/gitcode/llama.cpp && python convert.py /content/drive/MyDrive/gitcode/sample_data/llama-combined-ch-7"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 测试模型"
      ],
      "metadata": {
        "id": "SvGiYoEgAjdo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd /content/drive/MyDrive/gitcode/llama.cpp && chmod 777 main && ./main -m /content/drive/MyDrive/gitcode/sample_data/llama-combined-ch-7/ggml-model-f16.bin --color -f ./prompts/alpaca.txt -ins -c 2048 --temp 0.2 -n 256 --repeat_penalty 1.1\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HrLtfiv26BZQ",
        "outputId": "94dbf9b4-c7db-4667-e95c-9dc540bbffda"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "main: build = 649 (98ed165)\n",
            "main: seed  = 1688020234\n",
            "llama.cpp: loading model from /content/drive/MyDrive/gitcode/sample_data/llama-combined-ch-7/ggml-model-f16.bin\n",
            "llama_model_load_internal: format     = ggjt v1 (pre #1405)\n",
            "llama_model_load_internal: n_vocab    = 49954\n",
            "llama_model_load_internal: n_ctx      = 2048\n",
            "llama_model_load_internal: n_embd     = 4096\n",
            "llama_model_load_internal: n_mult     = 256\n",
            "llama_model_load_internal: n_head     = 32\n",
            "llama_model_load_internal: n_layer    = 32\n",
            "llama_model_load_internal: n_rot      = 128\n",
            "llama_model_load_internal: ftype      = 1 (mostly F16)\n",
            "llama_model_load_internal: n_ff       = 11008\n",
            "llama_model_load_internal: n_parts    = 1\n",
            "llama_model_load_internal: model size = 7B\n",
            "llama_model_load_internal: ggml ctx size =    0.07 MB\n",
            "llama_model_load_internal: mem required  = 14925.62 MB (+ 1026.00 MB per state)\n",
            ".\n",
            "llama_init_from_file: kv self size  = 1024.00 MB\n",
            "\n",
            "system_info: n_threads = 6 / 12 | AVX = 1 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 0 | AVX512_VNNI = 1 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | \n",
            "main: interactive mode on.\n",
            "Reverse prompt: '### Instruction:\n",
            "\n",
            "'\n",
            "sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.200000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000\n",
            "generate: n_ctx = 2048, n_batch = 512, n_predict = 256, n_keep = 21\n",
            "\n",
            "\n",
            "== Running in interactive mode. ==\n",
            " - Press Ctrl+C to interject at any time.\n",
            " - Press Return to return control to LLaMa.\n",
            " - To return control without starting a new line, end your input with '/'.\n",
            " - If you want to submit another line, end your input with '\\'.\n",
            "\n",
            "\u001b[33m Below is an instruction that describes a task. Write a response that appropriately completes the request.\u001b[0m\n",
            "> \u001b[1m\u001b[32mh请介绍一下抖音带货的运营策略。\n",
            "\u001b[0m抖音带货的运营策略可以从以下几个方面进行： 1.了解目标受众，根据受众的兴趣和需求来选择适合的产品；2.与优质的主播合作，提高直播间的影响力和吸引力；3.利用抖音平台的特点，如短视频引流和直播间互动等，将产品销售地带向用户；4.定期分析数据，了解用户对产品的反应和偏好，根据数据调整运营策略；5.与供应商和品牌建立良好的合作关系，以获得更优惠的佣金和商品资源。\n",
            "> \u001b[1m\u001b[32m 如何在直播间中选择和组合产品?\n",
            "\u001b[0m 在直播间中选择和组合产品时，可以根据直播间的特点和目标受众来进行决策。例如，如果直播间的主题是美妆类的，可以选择与之相关的产品进行组合；如果直播间的主题是健康食品类的，可以将相匹配的产品组合好，以吸引更多的观众。在和商品组合时，要确保产品之间没有直接的竞争，要根据观众的喜好和需求来选择，以提高销售和转化率。\n",
            "> \u001b[1m\u001b[32m如何做好停留模型来补充销量和转化？\n",
            "\u001b[0m要做好停留模型，可以采取以下措施：在直播\n",
            "> \u001b[1m\u001b[32m如何做好停留模型来补充销量和转化？\n",
            "\u001b[0m要在直播间中做好停留模型的补充，可以考虑以下方面：1.提供有吸引力的内容，给观众带来更多的价值和信息，增加他们停留的动机和感兴趣；2.与观众进行互动，回答他们的问题，建立良好的交流和互动关系；3.提供购买商品的便利，如设置商品链接、提醒观众下单等，增加转化率。4.提供与直播内容相关的其他内容，如教程、折扣促销等，吸引观众的关注和参与。\n",
            "> \u001b[1m\u001b[32m如何评估直播间的流量复盘？\n",
            "\u001b[0m评估直播间流量的效果可以从以下几个方面进行复盘：1.观众的留存和转化情况，看看观众停留的时长和实际购买的数量是否高于行业平均水平；2.直播过程中的互动和参与度，与观众的互动是否能够引起他们的兴趣和共鸣；3.直播内容的准确性和吸引力，比如选品和话术等，对观众的引导和满足程度是否达到预期；4.直播间的停留和转化数据，分析观众在直播间中停留的时长和实际购买的数量，了解直播效果的有效性。这综合的复盘结果，可以为直播间的优化和改进提供\n",
            "> \u001b[1m\u001b[32m请给出一个在抖音带货中促使观众购买的策略\n",
            "\u001b[0m要促使观众在抖音上购买产品，可以采取以下策略：在直播中选择具有吸引力和独特卖点的产品；通过与主播的互动，回答观众的问题，提供产品的详细解释和推荐；在直播中使用优惠活动，如限时特价、打折促销等，吸引观众下单购买；通过与观众的交流，了解他们的需求和喜好，为后续的跟进和售后提供良好的服务。\n",
            "> \u001b[1m\u001b[32m"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 如果想量化模型请参考\n",
        "https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/notebooks/convert_and_quantize_chinese_llama_and_alpaca.ipynb"
      ],
      "metadata": {
        "id": "JGcG9xxWAxRf"
      }
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "machine_shape": "hm",
      "provenance": [],
      "gpuType": "A100"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}